A Methodological Approach to Data Quality Management Supported by Data Mining

In this paper, we use the example of the car manufacturing domain to illustrate how data quality problems are addressed in practice today. We then propose a process model for data quality management (DQM) which meets the requirements of the current ISO 9001 standard and thus introduces a methodological, processoriented approach to DQM. Data mining methods that are typically applied to find interesting and previously unknown patterns in large amounts of data are being used to support several phases of this process model. The main idea behind the application of data mining methods is to deem data anomalies deviations from a ‘normal’ quality state. The primary advantage of our approach is an increased degree of automation and enhanced thoroughness and flexibility of DQM. 1. Data Quality Challenges in Automotive Manufacturing While product quality has always been a central focus at DaimlerChrysler, data quality has not yet received the attention it deserves. This does not mean that data quality has been completely neglected thus far, but most data quality initiatives have had solely a strong local focus. With the growing demand for the integration of distributed, heterogeneous databases into corporate warehouse applications, data deficiencies have become obvious, necessitating corporate-wide DQM to address data quality issues across system boundaries. This global data quality view implicates additional data quality perspectives and presents challenges regarding the related tools and methodologies. As correcting data already stored in a database system is much more expensive than setting up appropriate measures to prevent substandard-quality data to be entered into the systems, precautions should be given top priority. However, as real environments are complex, there will be always a need for measuring, monitoring, and improving the quality of data after it has initially been stored. This was one of the motivations for initiating a research project which focuses on the application of data mining technologies in the context of DQM. We have introduced the term Data Quality Mining, which we believe to have great potentials for both future research work and a successful transfer of data mining technology into daily work processes. In section 2, we propose a quality management system for data integration that meets the requirements of the current ISO 9001 standard. Section 3 presents a case study where a subset of these concepts has been applied to the QUIS (QUality Information System) database of the Global Services and Parts division of the Group using data mining techniques. Finally, we give an overview of related work and further research issues in section 4. Proceedings of the Sixth International Conference on Information Quality